Mixed-effects models for health care longitudinal data with an informative visiting process: A Monte Carlo simulation study.

Electronic health records are being increasingly used in medical research to answer more relevant and detailed clinical questions; however, they pose new and significant methodological challenges. For instance, observation times are likely correlated with the underlying disease severity: Patients with worse conditions utilise health care more and may have worse biomarker values recorded. Traditional methods for analysing longitudinal data assume independence between observation times and disease severity; yet, with health care data, such assumptions unlikely hold. Through Monte Carlo simulation, we compare different analytical approaches proposed to account for an informative visiting process to assess whether they lead to unbiased results. Furthermore, we formalise a joint model for the observation process and the longitudinal outcome within an extended joint modelling framework. We illustrate our results using data from a pragmatic trial on enhanced care for individuals with chronic kidney disease, and we introduce user-friendly software that can be used to fit the joint model for the observation process and a longitudinal outcome

The kidney failure risk equation:evaluation of novel input variables including eGFR estimated using the CKD-EPI 2021 equation in 59 cohorts

SIGNIFICANCE STATEMENT: The kidney failure risk equation (KFRE) uses age, sex, GFR, and urine albumin-to-creatinine ratio (ACR) to predict 2- and 5-year risk of kidney failure in populations with eGFR <60 ml/min per 1.73 m 2 . However, the CKD-EPI 2021 creatinine equation for eGFR is now recommended for use but has not been fully tested in the context of KFRE. In 59 cohorts comprising 312,424 patients with CKD, the authors assessed the predictive performance and calibration associated with the use of the CKD-EPI 2021 equation and whether additional variables and accounting for the competing risk of death improves the KFRE's performance. The KFRE generally performed well using the CKD-EPI 2021 eGFR in populations with eGFR <45 ml/min per 1.73 m 2 and was not improved by adding the 2-year prior eGFR slope and cardiovascular comorbidities. BACKGROUND: The kidney failure risk equation (KFRE) uses age, sex, GFR, and urine albumin-to-creatinine ratio (ACR) to predict kidney failure risk in people with GFR <60 ml/min per 1.73 m 2 . METHODS: Using 59 cohorts with 312,424 patients with CKD, we tested several modifications to the KFRE for their potential to improve the KFRE: using the CKD-EPI 2021 creatinine equation for eGFR, substituting 1-year average ACR for single-measure ACR and 1-year average eGFR in participants with high eGFR variability, and adding 2-year prior eGFR slope and cardiovascular comorbidities. We also assessed calibration of the KFRE in subgroups of eGFR and age before and after accounting for the competing risk of death. RESULTS: The KFRE remained accurate and well calibrated overall using the CKD-EPI 2021 eGFR equation. The other modifications did not improve KFRE performance. In subgroups of eGFR 45-59 ml/min per 1.73 m 2 and in older adults using the 5-year time horizon, the KFRE demonstrated systematic underprediction and overprediction, respectively. We developed and tested a new model with a spline term in eGFR and incorporating the competing risk of mortality, resulting in more accurate calibration in those specific subgroups but not overall. CONCLUSIONS: The original KFRE is generally accurate for eGFR <45 ml/min per 1.73 m 2 when using the CKD-EPI 2021 equation. Incorporating competing risk methodology and splines for eGFR may improve calibration in low-risk settings with longer time horizons. Including historical averages, eGFR slopes, or a competing risk design did not meaningfully alter KFRE performance in most circumstances

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Predicting Cardiovascular Disease Risk in Chronic Kidney Disease

Chronic kidney disease (CKD) is a long term condition in which glomerular filtration is reduced and/or proteinuria occurs. Cardiovascular risk factors in CKD are different to the general population and overall risk is higher too. Therefore, risk prediction tools for cardiovascular disease require specific validation in CKD.The Leicester City and County CKD (LCC) cohort of 17,248 anonymised individuals with CKD from 44 general practices was established. Cardiovascular events were identified from general practice and hospital records, and 2,072 cardiovascular events occurred during five years of follow-up.Risk factors for cardiovascular events in CKD were identified in a systematic review and a second systematic review updated a previous systematic review of risk prediction tools for cardiovascular events in CKD. Albumin, haemoglobin and phosphate were identified as risk factors to be consider for risk prediction tools in addition to factors included in general population risk prediction tools. Seven CKD-specific and six general population risk prediction models were identified. All models were developed using the Cox proportional hazards (‘Cox’) model.The LCC cohort was used to externally validate these models. Discrimination was worse and calibration suggested overprediction of risk in all models. The latter worsened as predicted risk increased. Some calibration improvement was achieved through Cox model baseline risk function re-estimation. There was no significant risk prediction improvement by including the variables identified in the systematic review. Sensitivity analysis suggested that the Cox model’s censoring at random assumption may have been violated in the risk prediction models due to the competing risk from death.Risk prediction models for cardiovascular events in people with CKD require improvement and updating to optimise risk prediction accuracy. Alternative methods, such as multi-state models, should be considered in future model development.</div

SCALE AND COSTS OF INTRODUCING CYSTATIN-C EGFR CALCULATIONS TO UK PRIMARY CARE

Introduction and Aims: The introduction of the CKD-EPI and cystatin-C testing has been recommended in recent CKD guidance. The cost impact of this has not been studied in a primary care CKD cohort. We evaluated the cost impact of these guideline changes for UK primary care. Methods: We analysed the baseline cohort of the PSP-CKD study, a large UK primary care CKD clinical trial. A laboratory calculated MDRD eGFR<60 ml/min/1.73m2 was the primary identifier for the cohort. Serum creatinine was also available. CKD-EPI (EPI) eGFR was recalculated from the serum creatinine. Costs were calculated using the criteria outlined in the 2014 NICE CKD guidance. Results: The records of 353,256 patients ≥18 years of age were analysed. 30,307 individuals had at least one MDRD eGFR3 months apart. Of those with a single MDRD eGFR60. Reclassification to EPI eGFR>60 was less likely in those with pre-existing second MDRD eGFR<60 (3.3% versus 11.1%, p<0.001).To assess the numbers eligible for cystatin-C assay, we further analysed the 20,166 individuals with confirmatory MDRD eGFR<60. 12,142 (60.2%) had EPI stage 3A CKD. Within the EPI 3A group 7,153 (58.9%) had stage A1 proteinuria and were therefore eligible for cystatin-C assessment. 3055 (25.2%) had not had a urinary protein assessment.Extrapolating nationally, ~2% of the adult population could be eligible for a cystatin-C assay. Assuming similar results across the UK’s adult population and at a conservative cost of £5.50 (€7.50) per test, the initial cost of implementing cystatin-C testing across the eligible CKD population would be £5.8 million (€7.9 million). This estimate does not include costs of interpretation and consequent changes to clinical management. Conclusions: Based on a large primary care CKD cohort, CKD stage reclassification using EPI is relatively infrequent, and occurs less often if a confirmatory MDRD eGFR is available. Up to 2% of the adult population and 60% of the stage 3A CKD population could require a cystatin-C measurement. Conservative cost estimates for this are high and further cost-benefit considerations of cystatin-C testing may be helpful. Table 1: Reclassification 1 MDRD<60 ≥2 MDRD<60 Whole Cohort EPI<60 8,689 18,909 27,598 EPI>60 1,086 (11.1%) 662 (3.3%) 1,748 (6.0%) No serum creatinine 366 595 961 Total 10,141 20,166 30,30

Mixed-effects models for health care longitudinal data with an informative visiting process: A Monte Carlo simulation study.

Electronic health records are being increasingly used in medical research to answer more relevant and detailed clinical questions; however, they pose new and significant methodological challenges. For instance, observation times are likely correlated with the underlying disease severity: patients with worse conditions utilise health care more and may have worse biomarker values recorded. Traditional methods for analysing longitudinal data assume independence between observation times and disease severity; yet, with healthcare data such assumptions unlikely holds. Through Monte Carlo simulation, we compare different analytical approaches proposed to account for an informative visiting process to assess whether they lead to unbiased results. Furthermore, we formalise a joint model for the observation process and the longitudinal outcome within an extended joint modelling framework. We illustrate our results using data from a pragmatic trial on enhanced care for individuals with chronic kidney disease, and we introduce user‐friendly software that can be used to fit the joint model for the observation process and a longitudinal outcome.The PSP‐CKD study was funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) East Midlands. Ongoing support for the study is funded by NIHR CLAHRC East Midlands and Kidney Research UK (Grant TF2/2015). JKB is supported by the MRC Unit Programme number MC_UU_00002/5. MJC is partially funded by the MRC‐NIHR Methodology Research Panel (MR/P015433/1)

Mixed-effects models for health care longitudinal data with an informative visiting process: A Monte Carlo simulation study.

Electronic health records are being increasingly used in medical research to answer more relevant and detailed clinical questions; however, they pose new and significant methodological challenges. For instance, observation times are likely correlated with the underlying disease severity: Patients with worse conditions utilise health care more and may have worse biomarker values recorded. Traditional methods for analysing longitudinal data assume independence between observation times and disease severity; yet, with health care data, such assumptions unlikely hold. Through Monte Carlo simulation, we compare different analytical approaches proposed to account for an informative visiting process to assess whether they lead to unbiased results. Furthermore, we formalise a joint model for the observation process and the longitudinal outcome within an extended joint modelling framework. We illustrate our results using data from a pragmatic trial on enhanced care for individuals with chronic kidney disease, and we introduce user-friendly software that can be used to fit the joint model for the observation process and a longitudinal outcome